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EXECUTIVE  SUMMARY 


This  document  is  a  template  for  a  Test  Plan  that  is  applicable  to  Joint 
Single  Integrated  Air  Picture  (SIAP)  System  Engineering  Organization  (JSSEO) 
Test  Events.  In  the  conduct  of  live  events,  Hardware-in-the-loop  (HWIL) 
simulation-driven  exercises,  or  constructive  models  and  simulation  analysis, 
data  will  be  collected  to  support  JSSEO  objectives.  The  main  purpose  of  a  test 
plan  is  to  ensure  that  the  objectives  have  been  clearly  defined  and  that  the 
correct  data  will  be  collected  to  support:  these  JSSEO  objectives,  experiments, 
and  follow-on  analyses. 

The  planning  documentation  for  a  particular  test  will  include  1)  the  Test 
Plan  outlined  in  this  template  and,  2}  a  Test  Readiness  Report,  which  is 
outlined  in  the  Standard  Test  Readiness  Report  Template  TR  2004-016.  The 
Test  Plan  establishes  the  test  objectives,  organizational  and  individual  roles 
and  responsibilities,  and  schedule  to  secure  all  resources  and  assets  required 
to  conduct  the  test.  The  Test  Readiness  Report  is  an  update  to  the  Test  Plan 
and  ensures  that  all  steps  necessaiy  to  commence  the  test  event  are  complete. 
The  Test  Readiness  Report  is  presented  to  the  designated  approval  authorities 
at  the  Test  Readiness  Review  with  Go/No-Go  criteria  established  for 
determining  readiness.  Approval  authority  signature  on  the  Test  Readiness 
Report  indicates  agreement  with  the  report  and  authorization  to  conduct  the 
test. 


The  test  planning  process  includes 

1.  Identifying  the  test  objectives 

2.  Ensuring  that  the  necessary  operational  conditions  are  met 

3.  Describing  the  roles  and  responsibilities 

4.  Describing  the  Verification,  Validation,  and  Accreditation  efforts  for 
simulations 

5.  Planning  post-event  analysis 

6.  Developing  a  schedule  for  planning,  executing,  analysis  and 
reporting. 

This  template  attempts  to  address  all  types  of  tests  envisioned.  However, 
certain  sections  are  not  applicable  to  all  types  of  tests,  so  this  template  should 
be  tailored  depending  on  the  type  of  event. 

In  the  Executive  Summary  of  the  Test  Plan,  provide  a  summary  of 
essential  information  regarding  the  testing/simulation  event.  Include 
high-level  objectives,  dates  and  location  of  the  event  and  how  the  results 
will  be  used. 
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STYLE  AND  FORMATTING  GUIDELINES 

This  test  plan  template  has  specific  style  types  built  into  it  to  allow 
common  formatting  across  test  plans.  Headings  are  defined  as  first  order, 
second  order,  third  order,  and  so  on;  or,  as  number  one,  number  two,  and 
number  three.  There  should  seldom  be  a  number  four  heading.  These  heading 
styles  are  called  “Heading  I,  Head  1,”  “Heading  2,  Head  2,”  “Heading  3,  Head 
3,”  and  “Heading  4,  Head  4.”  They  are  of  Bookman  Old  Style  font,  are  boldface, 
and  not  underlined.  Numbering  goes  as  1.,  1.1,  1.1.1,  etc. 

Figure  captions  use  the  style  “Caption.”  Table  titles  use  the  style  ‘Table 
Center.”  Appendix  titles  use  the  style  “Annex.” 

Updating  Table  of  Contents,  List  of  Figures,  List  of  Tables,  and  List  of 
Appendices  is  done  using  the  following  steps: 

a)  Identify  the  table  or  list  you  wish  to  update  and  right-click  inside  it. 

b)  Select  “Update  field.” 

c)  If  you  want  to  update  the  table  headers  AND  pages  numbers,  select 
“Update  entire  table.”  If  you  want  to  just  update  page  numbers,  select 
“Update  page  numbers  only.” 

In  accordance  with  the  JSSEO  configuration  management  policy,  the 
footer  of  the  document  should  have  the  following  format: 

WBS  number  Test  Plan  (Document  Control  Number)_Version 
NumberJSSEO_YYMMDD 
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1 . 1  Background 

Discuss  the  significant  events,  developments,  findings,  and/or 
management  decisions  that  led  to  this  test  being  conducted.  Reference  should 
be  made  to  previous  related  tests,  problems  found  during  operational  use, 
significant  historical  data,  major  focus  areas,  and  capabilities  of  the 
testing/ simulation  process,  as  appropriate.  Include  topics  such  as: 

1 .  Dates  of  Significant  Milestones 

2.  Origin 

3.  Process 

4.  Timeframe  and  Priorities 

5.  Location 

6.  Environment 


1.2  Purpose  of  Test 

Succinctly  state  the  top-level  purpose  of  the  test.  Identify  the  customer 
for  the  test  results.  Describe  the  final  product  of  the  test  (i.e.,  the  deliverable) 
and  how  the  customer  will  use  it. 

1.3  Scope  of  Test 

Identify  the  top-level  test  objectives,  hypotheses,  test  description,  and 
instrumentation.  Identify  the  participating  organizations,  test  elements,  and 
assessment  constraints  and  limitations. 
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2.  OVERALL  TEST  DESIGN 

2.1  Concept  of  Test  Operations 

Describe  the  general  test  approach  along  with  the  specific  methodologies 
and  techniques  used  by  the  test  team  to  plan,  organize,  and  manage  the  testing 
activity.  The  test  design  should  perform  the  following  functions: 

1 .  Structure  and  organize  the  approach  to  testing  in  terms  of  specific 
test  objectives; 

2.  Identify  key  measures  of  performance  (MOPs); 

3.  Identify  the  required  data  and  demonstrate  how  the  data  will  be 
gathered,  stored,  analyzed,  and  used; 

4.  Indicate  what  part  modeling  and  simulation  will  play  in  meeting  test 
objectives; 

5.  Identify  the  number  and  type  of  test  events  and  required  resources. 

2.2  Brief  Experiment  Description 

Specify  the  test  objectives,  events,  and  analysis  requirements. 

2.2.1  Experiment  Objectives 

Identify  objectives;  include  any  sub-objectives. 

2.2.2  Experiment  Hypothesis 

Identify  the  hypothesis  for  the  experiment  that  is  to  be  proved  or 
disproved. 

2.2.3  Attributes  and  MOPs  Measured 

Briefly  describe  the  parameters  or  outputs  that  will  be  used  to  evaluate 
system  performance.  MOPs  should  be  short  definitive  statements  beginning 
with  an  action  verb  (e.g.,  “measure”  or  “calculate”). 

2.2.4  Data  Management  and  Success  Criteria 

Summarize  data  and  instrumentation  requirements  and  data 
management  strategy.  A  detailed  Data  Management  and  Analysis  Plan  will  be 
provided  as  an  appendix  to  the  Test  Readiness  Report. 

For  the  data  requirements  listed,  identify  a  process  for  determining  that 
data  has  been  properly  collected.  (Did  the  test  go  as  planned?  Was  data 
collection  successful?  Is  data  quality  sufficient  for  post-event  analysis?  Is  more 
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or  supplemental  data  needed?  EOIs  identified  and  packaged  for  analysis? 

TORs  collected?  Media/tapes  set  for  next  operation?)  . 

2.2.5  Test  Methodology 

Describe  test  methodology  and  procedures  to  safely  and  efficiently 
acquire  the  appropriate  information  to  correctly  calculate  the  MOP. 

2.2.5. 1  Baseline  Experiment 

Describe  how  a  baseline  for  Critical  Experiments  will  be  established. 

For  example:  "The  first  set  of  runs  will  support  establishing  a  baseline  for 
the  E-2C  SLAP  performance.  Two  runs  will  be  taken  to  ensure  that  the  data 
between  the  two  runs  produces  similar  SLAP  results  and  that  the  process  is 
repeatable.  SLAP  attributes  will  be  calculated  for  these  runs  and  will  be  used 
as  the  standard  bearer  against  which  all  parametric  analysis  will  be  compared. 
It  is  expected  that  both  operator/ analyst  observations  and  the  SIAP  attributes 
will  reflect  a  minimum  of  differences  between  the  two  runs.  If  repeatable 
baseline  runs  are  not  achieved,  parametric  runs  will  not  be  conducted  until  the 
cause  for  lack  of  repeatability  is  determined  and  fixed." 

2.2.6  Requisites 

Identify  the  operational  context  required  to  properly  collect  the  data  for 
the  experiment.  Include  number  and  types  of  units  required.  Identify  Go/No- 
Go  criteria  for  conducting  the  event.  For  Models  and  Simulations,  identify 
specific  modeling  capabilities  that  are  essential  to  meeting  test  objectives. 

2.2.7  Data  Reduction  and  Analysis  Method 

Identify  the  data  reduction  process,  including  tools  used,  how  the  data 
will  be  used  and  by  whom,  and  how  the  data  will  be  provided  to  analysis  team. 
Describe  the  analysis  method,  including  description  of  tools/algorithms  for 
conducting  analysis. 

2.2.8  Analysis  Team 

List  the  analysis  team  lead  and  key  team  members.  Include  their  roles  in 
the  event  and  contact  information. 

2.2.9  Reporting  Schedule 

Include  the  schedule  for  conducting  the  analysis,  and  identify  any 
constraints  or  contingencies  on  delivering  the  report. 
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2.3  Additional  Experiments 

If  the  test  includes  multiple  experiments,  describe  the  first  critical 
experiment  in  section  2.2,  then  add  sections  2.3,  2.4,...,  2.n  as  necessary  for 
each  of  n  critical  experiments.  Follow  the  format  of  section  2.2  for  these 
additional  sections. 
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3.  MODELING  AND  SIMULATION  (M&S  Venues) 


3.1  Federation  Design 


Include  an  overview  of  the  components,  interfaces,  systems'  roles  in  the 
federation,  how  they  are  implemented,  and  any  support  elements  (Figure  1). 
List  each  federate  and  document  further  detail  for  each.  A  more  detailed 
discussion  of  federation  development  should  be  provided  in  a  separate 
appendix  to  the  Test  Readiness  Report. 


MLST-3 
■jtiUn  Pbu 


Test  c 
Control 
Federate 


hlaResults 


CRSD 

(Scenario  Driver) 

•  Red  Positional  d  ata  and 
•side  designation 
•Blot*  Positional  data  and 
side  designation 
•E-2C  positional  data 
•Provide  IFF  response  lo  ■ 
Interrogation  Request 


RTI  NG  version  1 .6:  Red  Air, 
and  response  messages,  and  E-2 


m  I  t.. ci  rogation 

>p.  mi  utli/ation.  monitoring. . . 


Support  tools 


Link  16  Emulation! 


Simulation 


HWIL 


Figure  1 .  Notional  Federation  Design 


3.2  Federate  Roles 

3.2.1  Federate  Name  (e.g.,  E-2C  Federate,  ESTEL) 

Provide  a  functional  description  of  the  Federates  that  will  be  used  during 
the  event. 

Role  in  Federation: 

•  State  federate’s  role(s)  in  the  federation. 
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•  For  example:  Simulates  E-2C  APS- 145  radar,  IFF 
interrogator/transponder,  and  navigational  systems. 

Constraints /Limitations 

•  State  federate's  constraints /limitations. 

Implementation : 

•  State  federate's  implementation. 

•  For  example:  AN/APS- 145  Radar  is  simulated  using  RISS. 

Federation  Verification,  Validation,  and  Accreditation  (W&A): 

•  State  pertinent  W&A  information. 

3.2.2  Support  Federates 

Identify  and  describe  support  federates  required  for  the  event.  For 
example: 

Test  Control 

•  Adapted  from  Navy  Infrastructure  (NI)  effort. 

•  Provides  federation  start/ stop  and  monitoring. 

hlaResults®  Version  2.0 

•  Commercial  product  to  collect  data  in  federation  and  play  back  data. 

3.2.3  Supporting  Tools 

Identify  and  describe  supporting  tools  that  are  required  for  the  event. 

For  example: 

Command,  Control,  Communication,  and  Intelligence  ( C3I )  Engineering  and 
Evaluation  Sustem  fCEES) 

•  Interoperability  tool  developed  by  Redondo  Systems,  Inc. 

•  Monitors  and  collects  TADIL  J  and  DIS  truth  data. 

Joint  Analysis  Displaij  Environment  (JADE) 

•  Three-dimensional  quick-look  tool  during  runs. 

•  Monitors  and  collects  TADIL  J  and  HLA  truth  data. 

•  Post-mission  three  dimensional  (3D)  replay  capability. 
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Tactical  Office  (TACO) 

•  Three-dimensional  quick-look  tool  during  test  runs. 

•  Monitors  and  collects  ECS,  ICC,  TADIL  J,  and  DIS  truth  data. 

•  Post-mission  2D  replay  capability. 

Performance  Evaluation  Tool  (PET) 

•  Metrics  evaluation  tool  developed  by  NSWC  Corona. 

•  Incorporates  ECS,  ICC,  TADIL  J,  and  HLA  truth  data. 

•  Post-mission  2D  replay  capability. 

•  Seamless  interoperation  with  ARCTIC. 

Automatic  Reconstruction  and  Correlation  Tool  for  Interoyerabilitu 
Characterization  (ARCTIC) 

•  Performs  Automatic  Truth  to  System  track  matching. 

•  Seamless  interoperation  with  PET. 

•  Flexible/tailorable  to  all  types  of  system  data. 

3.3  M&S  Verification,  Validation,  and  Accreditation  fW&A)  Process 

Verification,  Validation,  and  Accreditation  (W&A)  is  required  to 
determine  that  a  simulation  or  federation  of  simulations  is  appropriate  to  use 
for  a  particular  test  objective.  Models  and  simulations  must  be  accredited  for 
their  intended  use. 


The  test  plan  should  include  the  V&V  process  diagram  from  the  JSSEO 
Technical  Report  on  M&S  W&A  (TR  2003-006}  shown  in  Figure  2  that 
discusses  how  JSSEO  is  charged  with  providing  recommendations  to  decision 
authorities  in  the  Office  of  Secretary  of  Defense  (OSD)  and  Joint  Staff  on  how  to 
achieve  SIAP-related  requirements  across  all  Services  and  Agencies.  These 
recommendations  must  be  reviewed  by  the  affected  Services  and  Agencies  in 
order  to  achieve  consensus  on  their  implementation. 
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I  IPocument  Decision  O  Action 


Phase  1 : 
Designation 


Phase  2: 
Planning 


JSSEO 
goes  through 
designation 
process. 

AA  approves 

Designation 

Document 

Issue  Leads 
designate 

M&S  through 

MSD  writes 

VV&A  Activity 

IAP  or 

V&V  Plan 

Team  and  SAT 

Designation 

ESG  review  V&V 

Letter 

Plan 

Recommendations  for 
improving  MSS  (in 
Test  Report) 


Phase  3: 
V&V 

Execution 


SAT  ESG 

approves/disapproves 
V&V  Plan  (becomes 
Test  Plan  Appendix) 

MSD  executes  V&V 
Plan  with  VV&A 
Activity  Team 
oversight 


MSD  writes  V&V 
Report  (added  to  Test 
Readiness  Report 
I  Appendix) 


VV&A  Activity 
Team  and  SAT 
ESG  review  V&V 
results;  makes  a 
recommendation  to 
AA 


Phase  4: 
Accreditation 

AA  approves/ 
disapproves 
Test  Readiness 
Report  and 
Accredits  M&S 
MSD  conducts 
test/analysts 


MSD  writes  final 
Test  Report 


AA  =  Accreditation 
MSD  =  M&S  Developer 


Figure  2.  JSSEO  W&A  Process 
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The  W&A  process  includes  development  of  a  V&V  plan  for  each  of  the 
federates  and  the  overall  federation  itself.  The  purpose  of  the  W&A  Plan  is  to 
describe  how  the  test  team  applies  the  W&A  process  and  procedures  to  meet 
the  W&A  needs.  For  each  Federate  or  M&S,  there  will  be  a  section  dedicated 
to  its  specific  V&V  plan.  All  W&A  Plans  shall  reside  in  a  V&V  document 
separate  from  this  Test  Plan.  This  Test  Plan,  however,  shall  identify  (Table  1) 
those  federates  requiring  a  V&V  plan  and  the  corresponding  lead  for  each  plan 
Table  2  gives  a  schedule  of  the  W&A  process  for  this  test. 


Table  1.  Federates  Requiring  V&V  Plan 


Federate  requiring  V&V 
Plan 

Responsible  Party(ies) 

Primary 

Secondary 

Overall  Federation 

Utility  Player 

-  PATRIOT  Sim 
Interface 

-  CRS-D 

-  Tools  (TIAC,  JACE. 
CEES,  TACO) 

Utility  Player 

-  GTE  1 553 

-  DLS 

-  TIAC /H  LA  1  1 

Primary  Responoil '  J  | 

fy  % 

’iPi  ■  -y>  -  Ly  _  %■  ’ll  ■;  % 

•%  \  |  \ 1 

%r  -  fr  \  1  \  1  | 

Secondary  Responsible 
\  Party 

|  fv 

% 

m 

Indary  Responsible 
Party 

PATRIOT  Sim  Inter.  i  I 

-  GTE  X.25 

-  FMS-D 

^Jpmt  Jf  Responsible  Party 

Secondary  Responsible 
Party 

CRS-D 
-  CRS 

Primary  Responsible  Party 

Secondary  Responsible 
Party 

Table  2.  V&V  Schedule 


Date 

Action 

10  Mar  04 

All  V&V  plans  delivered  to  Maj.  Borows'  y  «  ft 

10-14  Mar  04 

V&V  Activity  team*  review  of’/'  V  \  \  f  %  borowsky  provides 
recommendations  *  »  /  f  \  \  fpproval  of  plans. 

19  Mar  04 

Status  update  in  H  \  .  Ing  preliminary  V&V 

reports.  Ill  1  I  1  I  \  \  !  \  .3. 

7  Apr  04 

Telecon  following  dr\  -  g.  ~i  |.  <v,y-yVreport.  Maj. 

Borowsky  provides  re  \  \  '  WSAT  ESG  prior  to  TRR  to 

accredit  or  not  accredit  #  ^ 

9  Apr  04 

Test  Readiness  Review  and  accreditation. 

*V&V  Action  team:  The  W&A  Action  Team  is  an  ad  hoc  team  of  SMEs, 
Model/Tool  developers /experts,  Service  representatives  and  other  specialists.  It 
will  normally  be  established  as  part  of  the  Test  Plan  Working  Group.  Provide 
team  members  and  representatives  from  each  organization  and  identify  their 
associated  organizations. 
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4.  TEST  SCHEDULE 

Present  the  overall  test  schedule,  in  accordance  with  the  project 
schedule,  from  event  kickoff  to  delivery  of  the  final  report.  Show  the  schedule 
of  events  in  list  or  timeline  format  (Gantt  chart,  see  Figure  3).  Include  any 
significant  pre-  and  post-test  requirements. 


ID 

"2" 

"F 

"4“ 

5 

6 

Y" 

~8~ 

....... 

10 

...... 

12 

"iF 

14 

15 

16 

if 

T9 

20 

21 

22 

23 

24' 


Task  Name  Fab  Mar  Apr  May  Jun  ju!  Aug  Sap  Oct  Ncv  Dec  Jan :  Feb  Mar  :  Apr  May :  Jun  Jui  Aug  Sep") 

SiAP- . ~  -  ■  "  . . . .  .  “ 


E2C  PILOT  FEDERATION  FOR  DATA 
»R0  TIME*  SYNCHRONIZATION 

E2  Sensor  Registration/Time  Sync  Error 
Development 

Define  E2C 


Implement  Sensor  Reg/Tim  Sync  Errors  & 
Interface 

Run  V&V  Scenarios 
Analyze  Results/ V&V  E2C  Sensor 
Interface  r.-:v  ■ 

Federation  Integration  (Scena  : 

install  and  Checkout  . "  ~j| 

Run  iniegratKin  Scenanos 
Accredit  Federation 
Develop  Test  Plan 

Generate  Draft  Test  Plan  &  SA* 

Deirver  first  Graft  of  Test  Plan 

.  -  .  > 

Review  Drafl  Test  Plan  a* 

Sign  Test  Plan 
Conduct  TRR 
Conduct  Event 

Conduct  Record  Test 
Analyze  &  Report 

Deliver  Quick-look  report 
SAT  Report  Analysis 
SAT  ESG  Analysis  Review 
Final  Report 


% 


% 


% 

% 


7/1  I 
7/1 


I  3/20 
1  3/20 


7/1 

7/1$”'  j 


7/12 

7/26 


;v 


m 

. . 


% 

\ 

» 


J8  f  ~ 

fcwf6 

y  10/18 

■  10/29 

i  j  1/4 

11/4  yy  11/15 

12/2oT 


■  12/20 

12/20  y  y  2/26 


Figure  3.  Notional  Schedule 

Because  the  Test  Plan  is  written  and  approved  well  in  advance  of  the  Test 
Readiness  Review,  many  of  the  tasks  necessary  to  commence  the  test  event  will 
be  incomplete  when  the  Test  Plan  is  approved.  For  those  tasks  to  be  completed 
after  the  Test  Plan  is  approved,  provide  a  closure  plan  in  sufficient  detail  to  be 
actionable,  and  identify  by  name  the  person  responsible  for  completing  the 
action. 
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5.  TEST  MANAGEMENT  AND  ORGANIZATION 

5.1  Roles  and  Responsibilities 

Provide  an  organizational  diagram  for  conducting  the  test.  Figure  4 
provides  a  notional  organization  of  an  event. 


JSSEO 

-  Test  Director 

-  Customer 

-  Event  Coordinator 

-  TP  and  DMAP 

-  SAT  SME  Support 
- CRS  Team 

-  VV&A  O'  rsi 


I  *  gi  1 


x-vheed  Martin  NSCC  PTC-1 
-  Site  Director/  Test  Conductor/ 
Data  Collection  Manager 


SME  Support  Staff 
-Platform  Analysts 
-Technical 
-Data  Collection 
-Critical  Experiment 
-Corona  Analysts 
-FOM 

-Site  Security 
-V&V 


Figure  4.  Notional  Organization  of  an  M&S  Event 


Discuss  the  specific  roles  and  responsibilities  for  each  organization.  For 
each  organization,  identify  key  point(s)  of  contact,  including  contact 
information. 


5.1.1  Customer  (e.g.,  JSSEO) 

The  customer  is  the  primary  user  of  the  test  results. 


The  customer: 

-  Has  primary  responsibility  for  marshalling  funding  resources 

-  Describes  the  expected  level  of  support  for  the  event 
Provides  some  resources  for  the  event 
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-  Coordinates  the  event 

-  Oversees  overall  planning,  conduct,  and  analysis  of  event 
Coordinates  test  plan  development  and  data  management  and 
analysis  plan 

-  Provides  guidance  on  critical  experiments  via  subject  matter 
experts 

-  Develops  the  CRS  excursion 

-  Provides  the  V&V  process 

-  Has  final  accreditation  authority  for  the  event. 

5.1.2  Test  Sponsor  Name  (e.g..  Joint  Theater  Air  and  Missile  Defense 
Organization,  JTAMDO) 

The  Test  Sponsor  is  a  resource  provider  and  endorses  the  scope  and 
goals  of  a  project  and  represents  the  test  throughout  the  management  process. 
The  Test  Sponsor  exercises  approval  authority  over  Test 
Objectives/Plans/Results. 

5.1.3  Application  Area  Manager  (e.g.,  Joint  National  Integration  Center, 
JNIC)  (M&S  Venues) 

The  Application  Area  Manager  provides  technical  environment  support 
services,  maintains  visibility  over  a  family  of  systems,  and  oversees  test 
requirements. 

The  Application  Area  Manager: 

-  Reviews,  evaluates  test  objectives,  plans,  analyses,  and  reports 

-  Participates  in  event  planning,  execution,  data  collection,  and 
analysis 

-  Provides  insight  for  other  test  activities  and  applications  to  the 
broader  testing  community 

5.1.4  Infrastracture/Technical  Manager  (e.g.,  Joint  Interoperability  Test 
Command  (JITC))  (M&S  Venues) 

The  Infrastructure /Technical  Manager  is  responsible  for  developing  the 
federation. 

The  Infrastructure/Technical  Manager: 

-  Develops  and  executes  a  V&V  plan  for  the  Utility  Player. 

-  Is  the  Configuration  Manager  with  the  responsibility  for  ensuring 
that  the  FOM  is  configured  properly  and  computer  program  versions 
used  are  documented 

-  Coordinates  and  maintains  the  Federation  Agreements  and 
coordinates  FOM  changes 
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-  Will  provide  technical  assistance,  if  requested,  to  issues  involving 
HLA  federate  design  or  the  RTI. 

5.1.5  Participating  Service(s)  (e.g..  Lower  Tier  Project  Office/Software 
Engineering  Directorate  (LTPO/SED)) 

Identify  the  participating  Service(s)  for  this  event. 

Participating  Services  will: 

-  Develop  test  procedures  for  conducting  experiments 

-  Conduct  V&V  of  their  federate  components  in  the  test  (M&S 
venues) 

-  Execute  test  runs 

-  Provide  Subject  Matter  Experts  to  ensure  test  objectives  are 
properly  addressed 

-  Develop  final  technical  reports  of  analysis  and  findings 

5.1.6  Supporting  Agencies  (e.g.,  Naval  Surface  Warfare  Center  (NSWC) 
Corona) 

Identify  roles  and  responsibilities  of  Supporting  Agencies. 

Supporting  Agencies: 

-  Ensure  that  the  test(s)  accurately  capture  program  attributes 
Provide  on-site  analysis,  as  necessary. 


5.1.7  SIAP  Analysis  Team  (SAT):  Executive  Steering  Group  (ESG)  and 

Other  Test  Representatives 

Identify  the  SAT  ESG  members  associated  with  the  subject  test  and  their 
intended  roles  and  responsibilities.  Include  statements  regarding  whether  the 
SAT  ESG  is  expected  to  provide  the  resources  necessary  to  plan,  execute,  and 
analyze  an  event. 

It  is  the  responsibility  of  SAT  members  to  ensure  the  right  tools  are 
brought  to  collect  necessary  data  and  perform  on-site  analysis. 

The  SAT  ESG  also  has  a  major  role  in  the  Verification,  Validation  and 
Accreditation  Process,  outlined  in  TR  2003-006  (M&S  venues).  It  will  be 
responsible  for  making  a  recommendation  to  accredit  the  federation. 
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5.1.8  SIAP  Common  Reference  Scenarios  (CRS)  Team 

Identify  the  CRS  team  that  will  be  responsible  for  developing  CRS 
excursions  that  reflect  the  needs  of  the  event. 

The  SIAP  CRS  Team  will: 

-  Develop  the  scenario  with  elements  and  formats  consistent  with  the 
FOM 

-  Ensure  the  scenario  contains  the  appropriate  requisites  to  conduct 
experiments 

-  Provide  data  required  to  conduct  test. 

5.2  On-site  Organization 

Identify  the  on-site  activity  management  personnel  and  their  roles. 
Identify  one  overall  leader  and  assistant  managers  (one  for  SIAP  Analysis  Team 
(SAT),  one  for  critical  experiments,  and  others  as  necessary  for  additional  test 
areas). 

Identify  the  SAT  on-site  objectives  such  as  mission  monitoring,  events  of 
interest  investigating,  and  root-cause  analysis  activities.  The  SAT  members 
should  participate  in  the  de-brief  process  and  interact  with  operators  whenever 
possible  to  address  SIAP  issues. 

Identify  the  Test  Observation  Report  (TOR)  Manager.  Discuss  the  TOR 
process  that  will  be  followed  for  capturing  SIAP-related  issues.  This  process 
should  include  adjudication  practices  to  be  used. 

Provide  a  table  that  lists  key  on-site  test  execution  and  analysis 
personnel,  their  roles,  the  system  or  agency  they  represent,  and  their  contact 
information.  As  appropriate,  identify  individuals  who  are  providing  analysis 
tools,  and  the  associated  logistics  information. 
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6.  REPORTING 

6. 1  Test  Readiness  Report 

The  Test  Readiness  Report  updates  the  Test  Plan  and  is  presented  to  the 
designated  approval  authorities  at  the  Test  Readiness  Review.  The  Test 
Readiness  Report  for  an  event  follows  the  guidelines  provided  in  the  Standard 
Event  Test  Readiness  Report  Template,  TR-2004- 16.  The  purpose  of  the  Test 
Readiness  Review  includes  1)  a  review  of  the  test  objectives,  methods,  data 
collection  and  analysis  plan,  individuals’  roles  and  responsibilities,  and  Go/No- 
Go  criteria,  and  2)  evidence  to  the  approval  authorities  that  all  preparations  for 
the  test  are  complete  and  the  test  can  be  completed  with  a  high  likelihood  of 
success.  Approval  signature  on  the  Test  Readiness  Report  indicates 
agreement  with  the  report  and  authorization  to  conduct  the  test. 

6.2  Quick-Look  Report 

Identify  the  organization (s)  responsible  for  producing  and/or  reviewing 
the  quick-  look  report  (s).  Quick-look  reports  shall  be  submitted  to  JSSEO 
within  30  calendar  days  of  completing  the  test  event.  Following  the  test  event, 
each  organization  submitting  a  quick-look  report  should  report  their 
preliminaiy  findings  as  they  relate  to  the  test  objectives.  Any  additional 
findings  of  significance,  especially  as  they  relate  to  the  SLAP  Attributes,  should 
also  be  reported.  Preliminary  conclusions  and  recommendations  as  they  relate 
to  the  test  objectives  should  be  included  as  appropriate. 

6.3  Technical  Report  Development 

Identify  organization (s)  responsible  for  producing  and/or  reviewing  the 
final  report.  Set  the  timeline  for  submission.  Establish  the  coordination 
process,  through  final  approval  authority.  State  expected  format  for  the  final 
report.  For  example:  '  A  technical  report  will  be  generated  within  90  days 
following  completion  of  the  E-2C  JDEP  event.  Generating  the  report  will  be  a 
collaborative  effort.  Final  signature  will  be  provided  by  JSSEO,  JTAMDO,  JNIC, 
JITC,  and  E-2C."  Table  3  gives  the  planned  schedule  for  the  reporting  process. 
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Table  3.  Reporting  Timeline  Requirements 


Description 

Responsible  Party(ies) 

Date 

Quick-look  report 

NLT  30  days  after  Test 
Event 

Review  of  final  results 

NLT  45  days  after  Test 
Event 

Review  and  comment 

NLT  60  days  after  Test 
Event 

Final  Technical  Report 
signed 

NLT  90  days  after  Test 
Event 
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APPENDIX  A:  ACRONYMS 


List  all  acronyms  in  the  document.  A  standard  set  of  frequently  used 
acronyms  is  provided  here  and  should  be  tailored  for  the  event  test  plan. 


AA  Accreditation  Authority 

ABT  Air-Breathing  Threat 

ACM/ACS  Automatic  Channel  Monitoring/ Automatic  Channel  Select 

AEW  Airborne  Early  Warning 

AGC  Automatic  Gain  Control 

ARCTIC  Automated  Reconstruction  and  Correlation  Tool  for 

Interoperability  Characterization 

ASCII  American  Standard  Code  For  Information  Interchange 


CCD 

CD 

CEC 

CID 

CNA 

COTS 

CRD 

CRS 

CRSD 


Common  Carrier  Device 
Compact  Disk 

Cooperative  Engagement  Capability 
Combat  Identification 
Center  for  Naval  Analyses 
Commercial  off  the  Shelf 
Capstone  Requirements  Document 
Common  Reference  Scenario 
Common  Reference  Scenario  Driver 


DCN 

DDM 

DEP 

DIS 

DISN 

DM 

DMAP 

DoDI 

DPCA 

DPG 

DR 

DX 


Document  Control  Number 
Data  Distribution  Manager 
Distributed  Engineering  Plant 
Distributed  Interactive  Simulation 
Defense  Information  Services  Network 
Data  Manager 

Data  Management  and  Analysis  Plan 
Department  of  Defense  Instruction 
Displaced  Phase  Center  Array 
Defense  Planning  Guidance 
Data  Recording/Data  Reduction 
Data  Extraction 


ESC/AW  Electronic  Systems  Center  (previously  referred  to  as  MASC) 

ESG  Executive  Steering  Group 

ESTEL  E-2C  Systems  Test  and  Evaluation  Laboratory 

FOM  Federation  Object  Model 

FoS  Family  of  Systems 

FTP  File  Transfer  Protocol 
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GII 

GIG 

GPS 

GRU 

GTE 

HLA 

HWIL 

IADS 

LAW 

ICC 

ICD 

ID 

IFF 

JCoCaC 

JDEP 

JIADS 

JITC 

JNIC 

JSSEO 

JTAMDO 

JTIDS 

KPP 

LTPO 

M&S 

MDA 

MIL- STD 

MOE 

MOP 

MS 

MSD 

MULTOTS 

NAVAIR 

N1 

NSWC 

OSD 


Group  II 

Global  Information  Grid 
Global  Positioning  System 
Gridlock  Reference  Unit 
Gateway  Terminal  Emulator 

High-Level  Architecture 
Hardware  in  the  Loop 

Integrated  Air  Defense  System 

In  Accordance  With 

Information  and  Coordination  Central 

Interface  Control  Document 

Identification 

Identification  Friend  or  Foe 

Joint  Council  of  Captains  and  Colonels 
Joint  Distributed  Engineering  Plant 
Joint  Integrated  Air  Defense  System 
Joint  Interoperability  Test  Command 
Joint  National  Integration  Center 
Joint  SLAP  System  Engineering  Organization 
Joint  Air  and  Missile  Defense  Organization 
Joint  Tactical  Information  Distribution  System 

Key  Performance  Parameter 

Lower  Tier  Project  Office 

Modeling  and  Simulation 
Missile  Defense  Agency 
Military  Standard 
Measure  of  Effectiveness 
Measure  of  Performance 
Microsoft 

Modeling  and  Simulation  Developer 

Multiple  Unit  Link  Test  and  Operations  Training  System 

Naval  Air  Systems  Command 
NAVAIR  Infrastructure 
Naval  Surface  Warfare  Center 

Office  of  the  Secretary  of  Defense 
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PC 

PET 

PO 

POC 

PPLI 

PU 

R2 

RISS 

RTI 

SAT 

SE 

SED 

SIAP 

SIF 

Sim/Slim 

SIPRNet 

SME 

SoS 

SPC 

SWIL 

STU 

TACCAR 
TADIL 
TAMD 
TAMD  CRD 

TD 

TDDS 

TF 

TIAC 

TIBS 

TIM 

TO 

TOM 

TOR 

TPWG 

TQ 

TRAP 

TSIU 

W&A 


Personal  Computer 
Performance  Evaluation  Tool 
Program  Office 
Point  of  Contact 

Precise  Participant  Location  and  Identification 
Participating  Unit 

Reporting  Responsibility 
Radar  IFF  Simulation  System 
Runtime  Infrastructure 

Single  Integrated  Air  Picture  Analysis  Team 

System  Engineer 

Software  Engineering  Directorate 

Single  Integrated  Air  Picture 

Selective  Identification  Feature 

Simulation/Stimulation 

Secret  Internet  Protocol  Router  Network 

Subject  Matter  Expert 

System  of  Systems 

Special  Programs  Center 

Software  in  the  Loop 

Secure  Telephone  Unit 

Time  Averaged  Clutter  Coherent  Airborne  Radar 
Tactical  Digital  Information  Link 
Theater  Air  and  Missile  Defense 

Theater  Air  and  Missile  Defense  Capstone  Requirements 
Document 

Test  Director  or  Tactical  Driver 
TRAP  Data  Dissemination  System 
Task  Force 

Theater  Air  and  Missile  Defense  Interoperability  Assessment 
Capability 

Tactical  Information  Broadcast  System 

Terminal  Input  Message 

Test  Objective 

Terminal  Output  Message 

Test  Observation  Report 

Test  Plan  Working  Group 

Track  Quality 

Tactical  Related  Application 
Tactical  System  Interface  Unit 

Verification,  Validation,  and  Accreditation 
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WAM 

Warfare  Assessment  Model 

WASP 

Wrap-Around  Simulator  Processor 

WG 

Working  Group 

WST 

Weapon  Systems  Trainer 

2D 

2  Dimensional 

3D 

3  Dimensional 
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APPENDIX  B:  SIAP  METRICS 

JSSEO  developed  a  set  of  attributes  (JSSEO  Technical  Report  2003-029) 
derived  from  TAMD  and  C1D  CRD  key  performance  parameters.  The  lest  plan 
should  describe  in  this  appendix  any  information  that  impacts  the  calculation 
of  the  SIAP  attributes  and  any  measures  of  performance.  All  JSSEO  tests 
should  include  a  SIAP  attributes  calculation.  Any  caveats,  limitations,  or 
changes  from  the  ordinary  to  compute  them  should  be  mentioned  here.  For 
reference,  the  qualitative  definitions  of  the  SIAP  attributes  are  provided  as 
follows: 

Completeness:  The  measure  of  the  portion  of  true  air  objects  that 
are  included  in  the  SIAP.  The  air  picture  is  complete  when  all 
objects  are  detected,  tracked  and  reported. 

Clarity:  The  measure  of  the  portion  of  the  SIAP  that  contains 
ambiguous  tracks  and/or  spurious  tracks.  The  air  picture  is  clear 
when  it  does  not  include  ambiguous  or  spurious  tracks. 

Continuity:  The  measure  of  how  accurately  the  SIAP  maintains 
track  numbers  over  time.  The  air  picture  is  continuous  when  the 
track  number  assigned  to  an  object  does  not  change. 

Kinematic  Accuracy:  The  measure  of  how  accurately  the  TAMD 
Family  of  Systems  (FoS)  reports  track  position  and  velocity.  The 
air  picture  is  kinematically  accurate  when  the  position  and  velocity 
of  each  assigned  track  agree  with  the  position  and  velocity  of  the 
associated  object. 

ID  Completeness:  The  measure  of  the  portion  of  tracked  objects 
that  are  in  an  identified  state.  The  ID  is  complete  when  all  tracked 
objects  are  in  an  identified  state. 

ID  Correctness:  The  measure  of  the  portion  of  tracked  objects  that 
are  in  the  correct  ID  state.  The  ID  is  correct  when  all  tracked 
objects  are  in  the  correct  ID  state. 

ID  Clarity:  The  measure  of  the  portion  of  tracked  objects  that  are 
unambiguously  identified.  The  ID  is  clear  if  no  tracked  object  is  in 
the  ambiguous  ID  state. 

Commonality:  The  measure  of  consistency  of  the  air  picture  held 
by  TAMD  FoS  participants.  The  air  picture  is  common  when  the 
assigned  tracks  held  by  each  participant  have  the  same  track 
number,  position,  and  ID. 
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The  actual  attribute  computations  will  be  automated  through  the  use  of 
the  Performance  Evaluation  Tool  (PET),  into  which  the  algorithms  for  the  SIAP 
attributes  have  been  encoded. 
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APPENDIX  C:  FEDERATION  DEVELOPMENT  PROCESS  (M&S  VENUES) 
Federation  Development  and  Execution  Process  (FEDEP) 

The  development  of  the  federation  designed  to  support  this  test  follows 
the  seven-step  FEDEP  process,  which  is  now  an  IEEE  standard  process.  This 
process  provides  the  framework  for  the  action  plan  and  development  schedule 
(Figure  C- 1).  The  steps  in  this  process  are  shown  in  Figure  C- 1 . 


Figure  C- 1 .  Federation  development  and  execution  process 


Step  1.  Define  Federation  Objectives 

The  first  step  in  this  process  is  to  clearly  define  the  federation  objectives. 
This  is  key  because  all  subsequent  steps  build  on  the  objectives.  This 
federation  is  designed  specifically  to  provide  the  environment  to  support  the 
stated  test  objectives  in  time  synchronization  and  data  registration 
experimentation. 
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Step  2.  Perform  Conceptual  Analysis 

The  next  step  is  to  define  characteristics  of  federates  and  the  federation 
needed  to  address  issues.  Of  particular  importance  in  this  test  is  credibility  of 
the  scenario  and  its  appropriateness  as  a  context  for  the  analysis  (sufficient 
numbers  and  positions  of  friendly  and  enemy  forces).  Equally  important  are 
the  characteristics  of  the  sensor  representation  in  terms  of  its  ability  to 
adequately  represent  the  actual  system,  and  the  inputs  needed  from  friendly 
forces  (PPLI,  IFF,  remote  tracks)  to  provide  the  environment  needed  for  the  test. 
These  federation  requirements  drive  the  selection  of  federates  and  the  W&A  of 
the  federation.  This  step  requires  active  participation  of  the  subject  matter 
experts  and  the  system  owners /proponents  since  it  is  dependent  on  a  sound 
understanding  of  the  problem  area,  the  substantive  issues  to  be  addressed  in 
the  test,  and  requirements  for  selection  of  the  representations  to  meet  the 
needs  of  the  test. 

Step  3.  Design  Federation 

The  next  step  is  to  identify  specific  federates,  develop  the  Federation 
Object  Model  (FOM)  for  the  federation,  define  federation  CONOPS,  and 
delineate  federate  upgrades  to  support  the  federation.  The  federation  design 
reflects  the  decision  of  how  to  satisfy  the  federation  requirements  with  specific 
federates,  scenarios  and  data  exchanges.  At  this  stage  it  is  almost  always 
necessary  to  return  to  steps  1  and  2.  It  may  be  necessary  review  the  objectives 
for  clarity  and  return  to  the  conceptual  analysis  with  more  detail  to  ensure  the 
requirements  for  the  federation  are  well  articulated  and  understood,  the 
federation  can  be  designed  to  meet  the  needs  of  the  user. 

Step  4.  Develop  Federation 

Next,  federate  owners  implement  support  for  the  FOM  and 
enhancements  in  federates  as  needed  and  test  individual  federates. 

Step  5.  Plan,  Integrate,  and  Test  Federation 

Incremental  testing  of  federation  capabilities  and  sets  of  federates  is 
completed  to  prepare  for  the  federation  execution  to  support  the  test. 

Step  6.  Execute  Federation  and  Prepare  Outputs 

The  test  is  then  conducted  using  the  federation  following  the  test  process 
and  procedures. 
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Step  7.  Analyze  Data  and  Evaluate  Results 

The  final  step  is  to  conduct  the  data  analysis,  evaluate  results,  and 
produce  the  final  report. 
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APPENDIX  D:  POINTS  OF  CONTACT 

Identify  names  of  participants  and  their  roles  in  the  event.  Provide  contact 
information. 


Table  D-l.  Participants  in  the  JDEP  Planning 


■jj 

s  wirmBrima— 

-  -  ' 

Last  name.  First  Name 

Company,  Office 
Symbol 

Table  D-2.  Test  Directors /Site  Test  Directors 


Pi  ■'site 

TD  /  Site  TO 

Ph.n« 

-  Fi 

ns 

For  example:  "Test 
Director  (Primary)" 

For  example:  "NAWC-AD 
(E2C)" 

For  example:  "Data 
Distribution  Manager" 

For  example:  "Data 
Collection  Manager" 

Table  D-3.  Data  Collection  Team 


System 

mmmm 

Title /Organization 

Name 

Phone 

Email 

For  example: 
"REPOSITORY" 

For 

example: 
"NAVSEA 
Corona,  CA" 

For  example:  "DX 
Coordinator.  NAVSEA 
Corona" 

Table  D-4.  Site  Leads/POCs 


Site 

Primary/ 

Alternate 

SttePOC 

Phone 

E-Mail  | 

For  example, 
"NAWC-AD  (E- 
2C)" 

Primary 

Alternate 
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Table  D-5.  Lead  Analysts 


POC 

Phone 

Email 
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